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H  With  the  advent  of  modem  electronics,  the  development  of  advanced  sensing 
technologies  has  led  to  an  explosion  in  the  amount  of  data  available  to  decision 
makers  in  many  fields.  Unfortunately,  despite  optimism  to  the  contrary,  the  ability 
of  humans  to  understand  and  respond  to  this  growing  flood  of  information  has 
not  kept  pace.  Lincoln  Laboratory  has  launched  a  multidisciplinary  research 
initiative  in  integrated  sensing  and  decision  support  (ISDS)  with  the  goal  of 
improving  decision  performance  across  a  variety  of  military  and  civil  applications 
by  augmenting  advanced  sensors  with  novel  information  exploitation  and 
management  techniques. 


The  history  of  remote  sensing  is  intimately  en¬ 
tangled  with  supporting  military  decision  mak¬ 
ing  through  the  processes  of  intelligence,  surveil¬ 
lance,  and  reconnaissance  (ISR).  While  the  principles  of 
military  intelligence  are  ancient  [1,  2],  the  application 
of  sensing  technology  to  reconnaissance  did  not  begin 
until  the  development  of  photographic  and  electronic 
surveillance  techniques  during  World  Wars  I  and  II  [3] . 
Since  then,  the  U.S.  Department  of  Defense  has  made 
significant  and  ongoing  investments  in  the  development 
of  sensing  capabilities  for  military  applications  [4],  To¬ 
day,  many  of  those  capabilities  have  found  equally  com¬ 
pelling  applications  in  the  civil  sector — from  the  use  of 
long-range  radar  for  air  traffic  control  to  the  develop¬ 
ment  of  earth-observing  satellites  for  environmental 
monitoring  and  research. 

As  remote-sensing  technology  has  advanced,  the 
amount  of  data  available  to  decision  makers  has  in¬ 
creased  at  an  ever-accelerating  rate.  While  gathering 
surveillance  and  reconnaissance  data  has  long  been  rec¬ 
ognized  as  the  key  enabler  for  lifting  the  “fog  of  war,” 
the  ability  to  manage  and  exploit  large  volumes  of  data 
from  multiple  sources  has  emerged  as  the  central  chal¬ 
lenge  for  supporting  decisions.  This  challenge  encom¬ 
passes  nearly  all  data-centric  decision  domains,  includ¬ 
ing  geopolitics,  law  enforcement,  medical  diagnostics, 
finance,  regulatory  affairs,  consumer  choice,  and  even 


recreational  pursuits  such  as  gambling  and  other  forms 
of  game  playing. 

Human  cognition  is  not  well  suited  to  accurately 
perceiving  large  amounts  of  information  and  possible 
courses  of  action — especially  when  that  information  is 
intrinsically  statistical  in  nature  [5,  6].  This  limitation 
gives  rise  to  the  temptation  to  attempt  complete  auto¬ 
mation  of  data-driven  decision  processes.  While  this 
approach  can  be  appropriate  for  certain  repetitive,  low- 
risk  applications,  most  decision  problems  demand  some 
degree  of  human  control  to  ensure  adaptability  to  unan¬ 
ticipated  circumstances. 

The  integrated  sensing  and  decision  support  (ISDS) 
initiative  at  Lincoln  Laboratory  seeks  to  identify,  char¬ 
acterize,  and  develop  automated  information  manage¬ 
ment  and  exploitation  technologies  to  augment  the 
unique  cognitive  capabilities  of  human  decision  makers. 
The  ISDS  initiative  operates  as  an  ensemble  of  parallel 
research  and  development  programs  across  a  variety  of 
application  domains  including  air  and  missile  defense, 
space  surveillance,  Navy  strike  and  special  operations, 
and  air  traffic  control  (including  weather  sensing  and 
collision  avoidance).  One  of  the  most  significant  chal¬ 
lenges  in  the  ISDS  initiative  has  been  identifying  a  com¬ 
mon  framework  in  which  to  understand  the  decision 
support  challenges  unique  to  each  of  these  sensor-cen¬ 
tric  applications.  Due  to  the  central  role  of  the  deci- 
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FIGURE  1.  The  observe-orient-decide-act  (OODA)  loop  highlights  the  multitude  of  factors  and  interactions  that  shape  de¬ 
cision  makers’  perception  of  unfamiliar  or  changing  situations  [7],  The  OODA  loop  has  been  an  influential  construct  for 
understanding  decision  making — especially  in  the  military  domain. 


sion  maker  in  each  application,  the  unifying  framework 
emerges  as  a  model  of  the  decision  process  itself. 

Modeling  the  Decision  Process 

Large-scale  human  endeavors,  such  as  military  cam¬ 
paigns,  involve  a  hierarchy  of  decision  makers,  from  the 
foot  soldiers  on  the  front  line  through  the  intermediate 
force  managers  to  the  high-level  theater  commanders. 
Each  increase  in  scope  compounds  the  complexity  of 
decision  making  as  more  situations  need  to  be  consid¬ 
ered,  more  evidence  needs  to  be  weighed,  and  more  op¬ 
tions  for  action  must  be  reviewed.  The  option  to  con¬ 
sider  more  situations,  to  gather  more  data,  to  perform 
more  analyses,  and  to  compare  more  alternatives  for 
action  render  decision  making  more  complex,  without 
necessarily  increasing  the  likelihood  of  successful  out¬ 
comes.  Although  decision  makers  at  all  levels  may  share 
the  same  overall  objective,  their  individual  perspectives 
and  experiences  will  independently  influence  their  ap¬ 
proach  to  the  decision  process — often  with  unexpected 
and  potentially  unfortunate  consequences. 

One  of  the  best-known  models  for  the  human  de¬ 
cision  process,  at  least  within  the  context  of  military 
applications,  is  the  observe-orient-decide-act  (OODA) 
loop  developed  by  U.S.  Air  Force  Colonel  John  Boyd 
[7].  The  key  elements  of  the  OODA  loop,  illustrated 
in  Figure  1 ,  are  the  variety  of  information  sources  that 
contribute  to  situational  understanding  and  the  mul¬ 


tiple  tiers  of  feedback  and  feed-forward  interactions. 
These  elements  emphasize  the  dynamic  and  oftentimes 
recursive  nature  of  collecting  and  assessing  information 
to  support  decision  making.  The  doctrine  of  “maneuver 
warfare”  [8]  recognizes  the  importance  of  taking  actions 
to  deliberately  disrupt  the  decision  cycle  of  an  adversary 
in  order  to  control  the  tempo  and,  ultimately,  the  out¬ 
come  of  a  conflict. 

The  OODA  loop  model  leaves  the  details  of  the  de¬ 
cision  process  itself  vague  in  the  recognition  that  deci¬ 
sion  making  can  be  highly  idiosyncratic.  To  lend  more 
predictability  to  the  process,  many  organizations  have 
instituted  formalized  decision-making  techniques.  The 
U.S.  Army,  for  example,  has  developed  the  Military 
Decision  Making  Process  (MDMP)  to  provide  a  reli¬ 
able  method  for  constructing  mission  plans  from  re¬ 
ceived  orders  [9].  The  MDMP,  summarized  in  Figure 
2,  is  centered  around  a  systematic  analysis  of  goals,  an 
intelligence  process  to  estimate  the  current  state  of  the 
battlefield,  the  development  of  potential  mission  plans 
by  the  operations  staff,  and  a  war-gaming  process  that 
estimates  the  possible  future  states  of  the  battlefield  as¬ 
sociated  with  each  plan.  The  decision  process  is  largely 
reduced  to  identifying  the  course  of  action  that  drives 
the  battlefield  closest  to  the  mission  goals. 

While  designed  to  explicitly  coordinate  the  activities 
of  multi-person  battle  staffs,  the  MDMP  identifies  a  key 
feature  of  the  decision  process  even  as  it  occurs  inside 
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the  minds  of  individual  decision  makers — the  selec¬ 
tion  of  actions  based  on  estimates  of  how  those  actions 
will  change  the  state  of  the  system  under  consideration. 
Usually,  the  system  to  be  modified  is  the  world  in  which 
we  live.  More  specifically,  the  system  is  a  particular  sub¬ 
set  of  this  world,  such  as  a  battlefield  for  military  deci¬ 
sions,  the  collections  of  personal  belongings  that  drive 
consumer  decisions,  or  more  fanciful  constructs  such 
as  game  boards  or  playing  fields  that  provide  contrived 
environments  for  recreational  decision  making.  Clearly, 
the  effectiveness  of  a  decision  process  based  on  simu¬ 
lation  of  potential  future  states  depends  on  developing 
an  accurate  model  for  the  evolution  of  the  system  and 
priming  that  model  with  an  accurate  representation  of 
the  current  system  state.  It  has  been  postulated  that  aug¬ 
menting  the  observe  and  orient  functions  of  the  OODA 
loop  with  the  projection  of  likely  future  states  describes 
the  process  by  which  humans  develop  and  maintain 
awareness  of  complex  situations  in  dynamic  environ¬ 
ments  [10]. 

Examinations  of  decision  making  in  a  variety  of  set¬ 
tings  reveal  that,  while  estimation  of  future  states  is  fun¬ 
damental  to  selecting  courses  of  action,  the  process  of 
generating  those  estimates  is  often  highly  synopsized. 
In  the  recognition-primed  decision  model,  outlined 
in  Figure  3,  both  the  recognition  of  situations  and  the 
selection  of  actions  can,  under  familiar  circumstances, 
be  replaced  with  prototypes  or  analogs  that  are  drawn 
from  the  decision  maker’s  past  experiences  [11].  Only 
when  faced  with  unfamiliar  or  anomalous  information 
does  the  decision  maker  need  to  develop  novel  courses 
of  action. 

This  synopsis  of  large  elements  of  the  decision  pro¬ 
cess  through  pattern  recognition  is  common  in  highly 
experienced  individuals  and  is  fundamental  to  making 
decisions  in  complex  environments  under  significant 
time  constraints  or  other  types  of  pressure.  As  elements 
of  the  decision  process  are  delegated  to  machine  auto¬ 
mation,  exploiting  situation  and  action  prototypes  fa¬ 
cilitates  the  development  of  decision  support  systems 
that  can  most  effectively  utilize  the  unique  cognitive 
capabilities  of  human  decision  makers  to  address  unan¬ 
ticipated  scenarios. 

As  part  of  the  ISDS  initiative,  we  have  developed  a 
simple  decision  model  that  captures  key  features  of  the 
work  described  in  this  overview.  The  ISDS  decision 
model,  illustrated  in  Figure  4,  focuses  on  the  processes 
internal  to  the  decision  maker  required  to  support  deci¬ 


sions  to  modify  the  environment;  such  modification  is 
achieved  through  the  employment  of  generic  actuators 
on  the  basis  of  information  derived  from  that  environ¬ 
ment  from  similarly  generalized  sensors.  The  core  of  the 
decision  process  is  the  simulation  of  potential  future  sit¬ 
uations  from  an  estimate  of  the  current  situation  and  an 
ensemble  of  courses  of  action  hypothesized  to  meet  the 
goals  of  the  decision  maker.  Finally,  situation  and  action 
templates  are  indicated  as  potential  synopses  of  multiple 
elements  of  the  decision  process. 

Although  ISDS  is  the  focus  of  a  specific  new  initia¬ 
tive  at  Fincoln  Faboratory,  the  problem  being  addressed 
is  shared  by  all  of  the  Faboratory’s  missions.  Thus  we 
can  reasonably  assume  that  the  research  described  here 
will  continue  to  identify  and  develop  common  tech¬ 
nologies  across  mission  domains,  so  that  in  the  future 
sensor  exploitation  services  developed  for  one  mission 
will  be  able  to  accommodate  the  needs  of  other  missions 
as  well. 
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FIGURE  2.  The  U.S.  Army’s  Military  Decision  Making  Pro¬ 
cess  (MDMP)  emphasizes  the  development  and  analy¬ 
sis  of  courses  of  action  (COA)  as  central  to  the  decision 
process  [9].  Intended  to  coordinate  the  actions  of  multi¬ 
ple  staff  officers,  the  MDMP  defines  a  formal  process  that, 
in  practice,  can  be  streamlined  as  circumstances  allow  or 
require.  Unlike  the  OODA  loop,  the  MDMP  does  not  describe 
an  explicit  observation  function;  instead,  it  focuses  on  the 
details  of  how  decisions  are  formulated  and  approved. 
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FIGURE  3.  The  Recognition-Primed  Decision  (RPD)  model  identifies  three  fundamental  scenarios  in  which  varying  degrees 
of  operator  recognition  can  simplify  large  portions  of  the  decision  process  [11].  In  the  first  scenario,  familiar  situations  can  be 
matched  with  typical  actions  to  facilitate  rapid  decisions  resulting  in  well-known  effects.  In  the  second  scenario,  an  unfamiliar 
situation  forces  the  decision  maker  into  a  diagnostic  process  in  order  to  develop  the  set  of  expectancies  required  to  select  a 
course  of  action  from  his  repertoire.  In  the  third  scenario,  a  satisfactory  action  may  not  exist  to  address  even  a  familiar  situa¬ 
tion.  In  this  case,  a  mental  simulation  process  may  be  required  to  develop  and  select  an  appropriate  course  of  action.  An  unfa¬ 
miliar  situation  requiring  a  novel  course  of  action  can  be  represented  by  a  combination  of  the  second  and  third  scenario. 


ISDS  in  Action 

To  illustrate  work  in  support  of  the  decision  process 
framework,  we  have  chosen  seven  examples  from  the 
civil  and  defense  missions  under  study  at  Lincoln  Labo¬ 
ratory.  A  recurring  theme  in  these  examples  is  the  sub¬ 
stantial  increase  in  the  use  of  automated  techniques  to 
supplement  the  manual  decision  support  processes  cur¬ 
rently  in  use.  Computer  modeling  and  simulation  are 
being  developed  to  provide  robust  situation  and  action 
templates,  as  well  as  to  supplement  large-scale  field  exer¬ 
cises  with  artificially  rendered  data  in  support  of  opera¬ 
tor  training  and  the  evaluation  of  decision  architecture 
effectiveness.  In  addition,  when  the  decision  time  ho¬ 


rizon  permits,  physics-based,  real-time  computational 
models  are  being  deployed  to  help  the  decision  makers 
visualize  the  potential  impact  of  courses  of  action. 

Evaluating  a  Decision  Support  Architecture 

Paula  Pomianowski  and  colleagues  discuss  the  instru¬ 
mentation  of  a  large,  operationally  relevant  field  exer¬ 
cise  (Silent  Hammer)  with  tools  to  relate  the  measures 
of  performance  of  the  deployed  decision  architecture  to 
the  overall  measures  of  effectiveness  of  the  field  opera¬ 
tion.  Among  the  automation  tools  under  test  in  this  ex¬ 
periment  was  a  self-synchronizing  Metadata  catalog  that 
was  used  to  manage  the  data  flow  between  analysts  and 
commanders  over  a  limited-bandwidth  communica- 
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FIGURE  4.  The  integrated  sensing  and  decision  support  (ISDS)  model  combines  key  elements  from  the  OODA,  MDMR 
and  RPD  models.  From  the  OODA  loop,  the  ISDS  model  explicitly  describes  the  observation  and  action  functions  as  the 
interactions  of  the  decision  process  with  the  external  environment  under  consideration.  From  MDMP,  the  ISDS  model 
decomposes  the  decision  process  to  indicate  the  role  of  simulation,  explicit  or  otherwise,  required  to  select  courses  of 
action  based  on  their  ability  to  drive  the  current  perceived  situation  to  a  more  desirable  future  state.  And  from  RPD,  the 
ISDS  model  recognizes  the  portions  of  the  decision  process  that  can  be  synopsized  by  the  recognition  of  appropriate 
situation  or  action  templates. 


dons  fabric.  One  important  outcome  of  this  effort  was 
the  characterization  of  the  dependence  of  collaboration 
in  the  battle  management  process  on  the  quality  and  rel¬ 
evance  of  the  available  surveillance  information. 

Factoring  Weather  into  Air  Traffic  Control 

Mark  Weber  and  colleagues  introduce  the  use  of  simu¬ 
lations  based  upon  physical  models  to  allow  a  decision 
maker  to  select  among  several  possible  courses  of  action. 
By  combining  computer-generated  weather  models  with 
candidate  aircraft  routing  options,  the  Lincoln  Labora¬ 
tory  team  has  demonstrated  how  to  extend  the  time  ho¬ 
rizon  of  air  route  traffic  planners  by  two  or  more  hours 
in  severe  weather  conditions.  By  using  the  extensive  data 
available  from  the  Corridor  Integrated  Weather  System 
in  the  eastern  half  of  the  United  States,  the  authors  have 
shown  how  the  air  traffic  management  system’s  deci¬ 
sion  making  can  become  “weather  informed.”  Special 
emphasis  has  been  placed  upon  the  development  of  op¬ 
erator  display  tools  that  help  the  traffic  planner  visual¬ 


ize  directly  how  various  routing  options  will  translate 
into  changes  in  airspace  operations  efficiency.  Another 
research  thrust  discussed  in  this  article  is  the  extent  to 
which  domain-specific  modeling  services  can  be  devel¬ 
oped  to  facilitate  interoperability  between  various  deci¬ 
sion-making  agencies:  in  this  case  the  development  of  a 
service-oriented  weather  decision  support  architecture  is 
proposed  as  an  enabler  for  consolidating  weather  fore¬ 
casting  systems  across  the  National  Airspace  System,  as 
well  as  providing  cross-mission  services  to  other  agen¬ 
cies,  such  as  the  Department  of  Homeland  Security. 

Averting  Midair  Collisions 

The  article  on  the  Traffic  Collision  Avoidance  System 
(TCAS)  by  Jim  Kuchar  and  Ann  Drumm  illustrates  the 
use  of  avoidance  maneuver  templates  to  provide  robust 
advisories  under  severe  time  pressure  to  resolve  airspace 
conflicts  between  aircraft.  This  worldwide-deployed 
system  uses  a  carefully  engineered  operator  interface  to 
insure  comprehensible  situation  awareness  prior  to  the 
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generation  of  resolution  advisories.  The  focus  of  the 
current  research  is  on  the  development  of  modified  ac¬ 
tion  templates  to  take  into  consideration  previously  un¬ 
anticipated  contingencies,  such  as  a  pilot  reversing  the 
sense  of  a  coordinated  resolution  maneuver.  The  authors 
have  extensively  analyzed  the  likelihood  of  such  events 
by  reviewing  data  recorded  by  the  Lincoln  Laboratory 
monitoring  program  in  the  Boston  area,  and  they  have 
developed  comprehensive  collision  risk  Monte  Carlo 
simulations  to  assess  the  impact  of  potential  changes 
to  the  decision  logic.  The  confidence  obtained  in  these 
results  has  led  the  authors  to  begin  investigating  the  im¬ 
pact  of  extending  TCAS  to  include  conflicts  with  un¬ 
manned  aircraft. 

Keeping  Satellites  Apart 

The  article  on  space  situational  awareness  by  Richard 
Abbot  and  Timothy  Wallace  illustrates  the  timely  iden¬ 
tification  of  potential  collision  risks  among  spacecraft  in 
geosynchronous  orbits.  This  task  entails  automating  the 
generation  of  situation  templates  based  upon  physical 
models.  The  most  challenging  cases  involve  situations 
wherein  at  least  one  of  the  spacecraft  is  not  under  active 
control.  This  case  has  led  the  authors  to  consider  Bayes¬ 
ian  networks  as  a  solution  framework. 

Simulating  Complex  Scenarios 

In  their  article  on  Virtual  Hammer,  Paul  Metzger  and 
colleagues  have  demonstrated  the  use  of  simulated  scene 
generation  and  sensor  models  to  extend  the  applicability 
of  expensive  field  exercises  under  controlled  variations 
in  the  decision  support  architecture.  The  authors  adopt 
an  approach  similar  to  that  used  in  evaluating  ballistic 
missile  defense  architectures  for  the  ISR  mission.  In  this 
case,  however,  they  focus  on  the  demands  introduced 
when  multiple  operators  compete  for  a  small  set  of  sen¬ 
sor  resources. 


Providing  Regional  Air  Defense 

In  the  final  piece  on  ISDS,  Curt  Davis,  Jim  Flavin,  and 
colleagues  provide  an  end-to-end  example  of  a  modern 
decision  support  environment,  in  enhanced  regional 
situational  awareness.  The  authors  illustrate  the  integra¬ 
tion  of  all  of  the  elements  of  distributed  sensing,  situ¬ 
ation  analysis,  action  templates,  and  decision  support 
for  regional  air  defense  under  stressing  timelines.  The 
architecture  also  demonstrates  the  use  of  effects-based 
feedback  from  warning  lights  and  interceptors  that 
help  clarify  pilot  intent  when  suspicious  behavior  is 
observed. 
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